A categorization of arguments for counting methods for publication and citation indicators

Most publication and citation indicators are based on datasets with multi-authored publications and thus a change in counting method will often change the value of an indicator. Therefore it is important to know why a specific counting method has been applied. I have identified arguments for counting methods in a sample of 32 bibliometric studies published in 2016 and compared the result with discussions of arguments for counting methods in three older studies. Based on the underlying logics of the arguments I have arranged the arguments in four groups. Group 1 focuses on arguments related to what an indicator measures, Group 2 on the additivity of a counting method, Group 3 on pragmatic reasons for the choice of counting method, and Group 4 on an indicator's influence on the research community or how it is perceived by researchers. This categorization can be used to describe and discuss how bibliometric studies with publication and citation indicators argue for counting methods

Validation of counting methods in bibliometrics

The discussion about counting methods in bibliometrics is often reduced to the choice between full and fractional counting. However, several studies document that this distinction is too simple. The aim of the present study is to give an overview of counting methods in the bibliometric literature and to provide insight into their properties and use. A mix of methods is used. In the preliminary results, a literature review covering 1970-2018 identified 29 original counting methods. Seventeen were introduced in the period 2010-2018. Twenty-one of the 29 counting methods are rank-dependent and fractionalized meaning that the authors of a publications share 1 credit but do not receive equal shares, for example harmonic counting. The internal and external validation of the counting methods are assessed. Three criteria for well-constructed bibliometric indicators - adequacy, sensitivity, and homogeneity - are used to assess the internal validity. Regarding the external validation of the counting methods, it is investigated whether the intentions in the studies that introduced the 29 counting methods comply with the subsequent use of the counting methods. This study has the potential to give a solid foundation for the use of and discussion about counting methods.Comment: Preprint: Author's manuscript submitted to the conference STI2020. Due to the Corona virus, STI2020 was postponed until September 2021. All submissions were returned to the authors before peer revie

Evaluation of unique identifiers used as keys to match identical publications in Pure and SciVal:a case study from health science

Unique identifiers (UID) are seen as an effective key to match identical publications across databases or identify duplicates in a database. The objective of the present study is to investigate how well UIDs work as match keys in the integration between Pure and SciVal, based on a case with publications from the health sciences. We evaluate the matching process based on information about coverage, precision, and characteristics of publications matched versus not matched with UIDs as the match keys. We analyze this information to detect errors, if any, in the matching process. As an example we also briefly discuss how publication sets formed by using UIDs as the match keys may affect the bibliometric indicators number of publications, number of citations, and the average number of citations per publication.  The objective is addressed in a literature review and a case study. The literature review shows that only a few studies evaluate how well UIDs work as a match key. From the literature we identify four error types: Duplicate digital object identifiers (DOI), incorrect DOIs in reference lists and databases, DOIs not registered by the database where a bibliometric analysis is performed, and erroneous optical or special character recognition. The case study explores the use of UIDs in the integration between the databases Pure and SciVal. Specifically journal publications in English are matched between the two databases. We find all error types except erroneous optical or special character recognition in our publication sets. In particular the duplicate DOIs constitute a problem for the calculation of bibliometric indicators as both keeping the duplicates to improve the reliability of citation counts and deleting them to improve the reliability of publication counts will distort the calculation of average number of citations per publication. The use of UIDs as a match key in citation linking is implemented in many settings, and the availability of UIDs may become critical for the inclusion of a publication or a database in a bibliometric analysis

Applying the Leiden Manifesto principles in practice:commonalities and differences in interpretation

p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 10.0px 'Times New Roman'} The Leiden Manifesto (LM) is changing how we think about and use metrics [1]. Bibliometric evaluation is explained as a combination of quantitative and qualitative methods, allowing the use of different metrics, disciplinary knowledge and research performance strategies. Both bibliometricians and consumers of bibliometrics are encouraged to communicate and use the LM principles to acknowledge what they know and do not know, what is measured and what is not measured, thus legitimizing the use of the metrics. However, in our previous study, we observed that it is unclear how the LM principles should be interpreted [2, 3]. We suspect that subjective interpretations of the principles do not correlate. To investigate the reliability and validity of the LM, the present study presents a systematic review of bibliometric reports that apply the LM principles. Reports are retrieved from the LM blog [4], Scopus, Web of Science and Google Scholar. Each principle and its interpretation is coded in NVivo, whereafter we explore the degree of agreement in the interpretations across the reports. We find that for some principles, e.g. principle 1, the interpretations are well aligned. For other principles, e.g. principle 3, the interpretations differ but may be seen as complementary. We also observe that interpretations can overlap and thus the redundancy of the principles needs to be further investigated, e.g. principle 3 and 6. We conclude that at least for some of the LM principles, the reliability appears weak as the range of interpretations are wide, however complementary. Furthermore, some of the interpretations are applied for more principles, which may point to weak validity. Further research on the reliability and the validity of the LM will be essential to establish guidance in implementing the LM in practice. </p

Stå på mål for den gode bibliometriske analyse

Når vi som bibliometrikere anvender eller rådgiver om bibliometri, oplever vi, at det kan være svært at efterleve internationale standarder for god praksis. Bibliometriske indikatorer anvendes i stigende grad i vurderingen af institutioners og forskeres performance. Bibliometri anvendes også i forbindelse med bevillinger, beslutningsprocesser og ved generel vurdering af forskningskvalitet, især nu hvor de offentlige investeringer i forskning mindskes, og prioritering er nødvendig. Derudover er det blevet mere påtrængende at vise omverdenen den samfundsmæssige nytte af forskning